Variable Albedo Surface Reconstruction from Stereo and Shape from Shading

We present a multiview method for the computation of object shape and re ectance characteristics based on the in tegration of shape from shading SFS and stereo for non constant albedo and non uniformly Lambertian surfaces First we perform stereo tting on the input stereo pairs or image sequences When the images are uncalibrated we recover the camera parameters using bundle adjustment Based on the stereo result we can automatically segment the albedo map which is taken to be piece wise constant using a Minimum Description Length MDL based met ric to identify areas suitable for SFS typically smooth textureless areas and to derive illumination information The shape and the illumination parameter estimates are re ned using a deformable model SFS algorithm which it erates between computing shape and illumination param eters Our method takes into account the viewing angle dependent foreshortening and specularity e ects and com pensates as much as possible by utilizing information from more than one images We demonstrate that we can ex tend the applicability of SFS algorithms to real world sit uations when some of its traditional assumptions are vio lated We demonstrate our method by applying it to face shape reconstruction Experimental results indicate a sig ni cant improvement over SFS only or stereo only based reconstruction Model accuracy and detail are improved especially in areas of low texture detail Albedo informa tion is retrieved and can be used to accurately re render the model under di erent illumination conditions